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Bit arrays, or bitmaps, are used to significantly speed up set operations in several areas, such as data warehousing, information 
retrieval, and data mining, to cite a few. However, bitmaps usually use a large storage space, thus requiring compression. Never- 
theless, there is a space-time tradeoff among compression schemes. The Word Aligned Hybrid (WAH) bitmap compression trades 
some space to allow for bitwise operations without first decompressing bitmaps. WAH has been recognized as the most efficient 
scheme in terms of computation time. In this paper we present Concise {Compressed 'n' Composable Integer Set), a new scheme 
that enjoys significatively better performances than those of WAH. In particular, when compared to WAH, our algorithm is able 
to reduce the required memory up to 50%, by having similar or better performance in terms of computation time. Further, we 
show that Concise can be efficiently used to manipulate bitmaps representing sets of integral numbers in lieu of well-known data 
structures such as arrays, lists, hashtables, and self-balancing binary search trees. Extensive experiments over synthetic data show 
the effectiveness of our approach. 
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■1. Introduction 

The term bit array or bitmap usually refers to an array data 
structure which stores individual bits. The main reason for 
adopting bitmaps is represented by their effectiveness at ex- 
ploiting bit-level parallelism in hardware to perform operations 
quickly. A typical bit array stores fcxw bits, where w is the word 
size, that is the number of bits that the given CPU is able to 
manipulate via bitwise instructions (typically 32 or 64 in mod- 
ern architectures), and k is some non-negative integer Bitmaps 
made up of n bits can be used to implement a simple data struc- 
ture for the storage of any subset of {1, 2, . . . , n). Specifically, if 
the i-th bit of a bitmap is "1" then the integer / is within the inte- 
ger set, whereas a "0" bit indicates that the number correspond- 
ing to its position is not in the set. This set data structure uses 
fn/wl words of space — padding with zeros is a usual choice. 
Whether the least significant bit (the "rightmost" in the typi- 
cal bitmap representation) or the most significant bit (the "left- 
most" one) indicates the smallest-index number is largely irrel- 
evant, but the former tends to be preferred and will be adopted 
throughout all the examples proposed in this paper. Therefore, 
if we want to compute the intersection or the union over inte- 
ger sets represented by bitmaps of length n, we require [n/w] 
bitwise AND/OR operations each. 

Because of their property of leveraging bit-level parallelism, 
bitmaps often outperform many other data structures (e.g., self- 
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balancing binary search trees, hash tables, or simple arrays or 
linked lists of the entries) on practical data sets, even those 
which are more efficient asymptotically. However, bitmaps show 
some drawbacks as well: they are wasteful in both time and 
space when representing very sparse sets — that is, sets contain- 
ing few elements compared to their range. For such applica- 
tions, compressed bitmaps should be considered instead. Clas- 
sical compression algorithms introduce a computation overhead 
that may limit the benefits of using bitmaps. For example, well- 
known algorithms such as Deflate |1] effectively reduce the 
memory footprint, but performing set operations requires data 
to be decompressed, hence drastically increasing the computa- 
tion time 1 2]. That is why compression schemes that allows for 
bitwise operations without first decompressing bitmaps are to 
be preferred, at the cost of having a memory footprint higher 
than other compression schemes. In this scenario, the Word 
Aligned Hybrid (WAH) bitmap compression algorithm is cur- 
rently recognized as the most efficient one, mainly from a com- 
putational perspective |2]. It has been first proposed to com- 
press bitmap indices of DBMS, but subsequent applications in- 
clude visual analytics ([stl and data mining iQl, to cite a few. It 
uses a run-length encoding, where long sequences of O's or I's 
require a reduced number of bits by only storing the length of 
the sequences in place of the corresponding bit strings. WAH 
allows for set operations to be efficiently performed over the 
compressed representation. This property is guaranteed by the 
alignment with the machine word size. Figure [T] graphically 
explains what "alignment" means. Without loss of generality, 
suppose that words are made up of 32 bits. First, we logically 
partition the bitmap to compress into blocks of 3 1 bits, namely 
the word size minus one. In turn, sequences of consecutive 31- 
bit blocks containing all O's or all I's are being identified. The 
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Figure 1 : Word-aligned run-length encoding 



compressed form is created as follows: if a 31 -bit block con- 
tains both O's and 1 's, it is stored in a 32-bit word referred to 
as literal word, where the leftmost bit is set to 1. Otherwise, 
sequence of homogeneous 3 1 -bit blocks of O's or 1 's are stored 
in a single 32-bit word referred to as fill word, where the first 
(leftmost) bit is 0, the second bit indicates the fill type (all O's or 
all 1 's) and the remaining 30 bits are used to store the number of 
31 -bit blocksQ Bitwise operations are fast because performing 
AND/OR over compressed bitmaps corresponds to performing 
AND/OR over 32-bit literal pairs (with just one CPU instruc- 
tion), while sequences can be easily managed due to the same 
block granularity of literals. 

This paper proposes a new compression scheme. Concise 
(Compressed 'n' Composable Integer Set), that outperforms 
WAH by reducing the size of the compressed bitmaps up to 
50%, without affecting the performance of bitwise operations. 
For some widespread data configurations. Concise is even faster 
than WAH. Concise is based on the observation that very sparse 
bitmaps (that is, when there are few set bits followed by long 
sequence of unset bits) can be further compressed with respect 
to the WAH approach. Specifically, if n is the number of bits 
that equals 1 in a sparse uncompressed bitmap, a WAH-based 
compressed bitmap requires about 2n words: one word for the 
literal that contains the set bit, and the other word for a literal 
that counts the subsequent unset bits. In fact, 2n is the upper 
bouncQfor the size of bitmaps with n set bits fllsl]. To achieve 
a better compression ratio on sparse bitmaps. Concise intro- 
duces the concept of "mixed" fill word. In particular, we allow 
to store the sequence length and the literal word made up of 
only one set bit within a single word of 32 bits. This reduces 
the worst case memory footprint to n words. Since n words is 
the minimum amount of memory required to represent n inte- 
gral values with classical data structures (e.g., with an array). 
Concise is always more efficient than other structures in terms 
of memory footprint. As for computation time, we show that it 
also outperforms efficient data structures such as hashtables or 
self-balancing binary search trees on set operations. 

The remainder of the paper is organized as follows. The 



'in the paper of Wu et al. |2], the most significant bit is complemented with 
respect to the example of Figure \i\ that is literals start with and fills start 
with 1 . Though this does not change the semantic of the approach, we use 
the configuration of Figure \l\ since it reflects the proposed implementation of 
Concise. 

^Apai1 from a few additional data required to manage the actual coding of 
the algorithm, such as the number of the leftover bits within the last literal word. 



following section offers a detailed description of the Concise 
algorithm. The benefits of the proposed algorithm are then ex- 
perimentally demonstrated in Section[3] Finally, Section|4]pro- 
vides concluding remarks. 

2. Concise Algorithm 

Figure|2]shows an example of CONCiSE-compressed bitmap 
made up of 5 words. Words #0, #3, and #5 are literal words 
where, similar to WAH, the leftmost bit indicates the block type 
('1'), while the remaining bits are used to represent an uncom- 
pressed 31 -bit block. Words #1, #2, and #4 are fill words: the 
first (leftmost) bit is the block type ('0'), the second bit is the 
fill type (a sequence of O's or I's), the following 5 bits are the 
position of a "flipped" bit within the first 31 -bit block of the 
fill, and the remaining 25 bits count the number of 3 1 -blocks 
that compose the fill minus one. When position bits equals 
(binary '00000'), the word is a "pure" fill, similar to that of 
WAH. Otherwise, position bits indicate the bit to switch (from 
to 1 in a sequence of O's, or from 1 to in a sequence of I's) 
within the first 31 -bit block of the sequence represented by the 
fill word. That is, 1 (binary '00001') indicates that we have to 
flip the rightmost bit, while 31 (binary '11111') indicates that 
we have to flip the leftmost one. If we consider bitmaps as a 
representation of integer sets, in Figure |2] Words #2 indicates 
that integers in the range 94-1022 are missing, but 93 is in the 
set since position bits say that the first number of the "missing 
numbers" sequence is an exception. 

This approach allows to greatly improve the compression 
ratio in the worst case. Indeed, instead of having 2n words in 
the WAH-compressed form of n sparse integers, we only re- 
quire n words in the CONCiSE-compressed form for the same 
integer set. This way. Concise bitmaps always require less 
amount of memory than WAH bitmaps. As for performance, 
in the next section we will show that our proposal not only do 
not increase the computation time, but in some cases can also 
speed up operations thanks to the reduced number of words to 
manipulate. Since we have 25 bit for representing the length 
(minus one) of sequences of homogeneous bits, the maximum 
representable integer is 31 x 2^5 -i- 30 = 1 040 187 422 ^ 2^°, 
that is half of the positive integers that can be represented in a 
two's complement representation over 32 bits. 

Due to space limitation, in Figure [3] we only describe with 
pseudo-code the three main parts of the proposed implemen- 
tation of the Concise scheme. These parts are sufficient to 
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Figure 2: Compressed representation of tlie set {3, 5, 31-93, 1024, 1028, 1 040 187 422). The word #0 is used to represent integers in the range 0-30, word 
#1 for integers in 31-92, word #2 for integers 93-1022, word #3 for integers 1023-1053, word #4 for integers 1054-1 040 187 391, and word #5 for integers 
1040187 392-1040187 422. 



have a complete comprehension of the algorithm. However, the 
complete source code is available on SourceForge since January 
2010. The Concise algorithm has been coded in Java. More 
details about the actual implementation of Concise, as well 
as the code used for the comparative analysis, can be found 
at http : / / sourcef orge . net/pro j ects/ concis 60 In Fig- 
ure |3]the following notation has been adopted: 

• "I" indicates the bitwise OR operator, "&" the bitwise 
AND, "~" the bitwise NOT, and "«;" means the left-shift 
operator. 

• SOOOOOOOh is an instance of a 32-bit word expressed in 
hexadecimal notation. That is, SOOOOOOOh indicates a 32- 
bit word with the highest-order (leftmost) bit set to 1 and 
all other bits set to 0. 

We also requires additional bit operations that can be efficiently 
performed in most machines. In particular: 

• BitCount(«) counts the total numberof set bits in the bi- 
nary representation of the specified integral value n — also 
known as population count or Hamming weight. It is na- 
tively provided by some processors via a single instruc- 
tion to calculate it (i.e., PQPCNT of Intel SSE4 instruc- 
tion set For processors lacking this feature, there 
are efficient branch-free algorithms which can compute 
the number of bits in a word using a series of simple bit 
operations 

• TRAiLiNGZEROS(n) counts the number of O's following 
the lowest-order (rightmost) bit in the binary representa- 
tion of the specified integral value 

• CONTAiNSONEBiT(n) checks whether the given number 
contains only one set bit, and it can be efficiently per- 
formed by computing n & (« - 1) = 0. 

• n X 31 can be performed by doing (« <sc 5) - n. Several 
bit hacks exist to efficiently compute n mod 3 1 . 



'The first release can be downloaded at 
[http : //concise . svn. sourcef orge .net/vi6wvc/concise?view=rev&revi 
Following versions can present some improvements, hence being slightly dif- 
ferent from the algorithm described in this paper 

'^For more details about bit hacks, see, for example, 
^ttp : //graphics . Stanford. edu/~seande r/bithacks .htmr] 



3. Algorithm Analysis 

In this section we report on the results of a comparative 
analysis among our CONCISE implementation, WAH, and some 
classical data structures to manage integer sets. The testbed 
was represented by a notebook equipped with an Intel Core^'^2 
Duo CPU P8600 at 2.40 GHz and 3GB of RAM. All algorithms 
were coded in Java SE6. Since we did not find any reliable 
implementation of WAH in Java, testing WAH was performed 
by "switching off"' the possibility of having "mixed" fill words 
in our implementation of Concise. This also assures that dif- 
ferences in performances are mainly due to the compression 
schema, and not to the given implementation. As for other data 
structures, we used the classes provided by the Java package 
java.util. Pseudo-random numbers were generated through the 
algorithm described in due to its provable good uniform dis- 
tribution and very large period. 

Table [T]reports some characteristic about the memory foot- 
print of the data structure under analysis. For each structure, we 
report the number of bytes required to store each integral num- 
ber, whether the structure allows for duplicate elements, and if 
the items are kept sorted or not. Concise is the more efficient 
data structure in terms of memory occupation. In fact, classical 
structures incur an additional linear space overhead for point- 
ers, while WAH requires twice the memory of Concise when 
both algorithms are able to compress data — that is, in presence 
of sparse datasets. 

Figure |4] reports on experimental time-space analysis re- 
sults. In our experiments, we generated sets of 10'' integers, 
ranging from to a variable maximum value max. Within this 
range, numbers were generated at random according to two dif- 
ferent probability distributions: uniform and Zipfian. In particu- 
lar, at each generation of a pseudo-random number a € [0, 1 ), in 
uniform sets an integer corresponding to [a x maxj was added, 
where max - 10^ /d by varying d (the "density") from 0.005 
to 0.999. Numbers were generated till reaching the cardinal- 
ity of 10^. Similarly, in Zipffan sets, at each number genera- 
tion, an integer corresponding to [max x a'^i was added, where 
rpax^e [1.2 X 10^, 6 X 10^]. In this way, we generated skewed 
such that most of the integers were concentrated to lower 
values, while numbers with high values were very sparse. The 
reason for a Zipffan distribution is that, according to the Zipf 's 



procedure AppEND(words[], top, max, i) 
{first append] 
if words [■] is empty then 
/ ^ L173IJ 
if / = then 

top <- 
eise if / = 1 then 
top <- 1 

words[0] ^ SOOOOOOOh {literal} 
else 
top ^ 1 

words[0] - 1 {fill] 
end if 

words [top] ^ SOOOOOOOh | (1 « (/ mod 31)) 
max <— (■ 

return words[ ], top, max 
end if 



18: b <— i - max + (max mod 31) {position of next bit to set] 



{cfieck if zeros are required before the new word] 
if > 31 then 

/«-Lfo/31J 

if / = then 

top <- top + 1 {Just add a new word to set ttie bit] 

else 



{add a O's before ttie new word] 

if CoNTAiNsONEBiT(words[top]) then 

{merge tlie previous word] 

words[top] <- (1 + TRAiLiNGZEROS(words[top])) « 25 I / 
top «- top + 1 



else 

top <— top + 2 
if / = 1 then 

words [top - 1] <- 
else 

words [top - 1] <- 
end if 
end if 
end if 

{prepare ttie new word] 
b b mod 31 
words [top] ^ SOOOOOOOh 
end if 

words[top] <- words[top] | (1 ■ 
max / 

CoMPRESs(words[-],top) 
return words[-], top, max 
end procedure 



SOOOOOOOh {literaf] 
f - 1 {fill] 



I {set ttie bit] 



(a) Append of a new integer i that is greater than the maximal appended integer max. It checks whether the bit to set is within the last literal, or if we need to add a 
sequence of O's before setting the bit. It also returns top, namely the index of the last word of words. 



procedure CoMPRESs(words[-],top) 

if top = then {cfieck if ttie set is empty] 

return words[ ],top 
end if 

ifo *~ words [top] = SOOOOOOOh {last word all O's] 
If I <- words[top] = FFFFFFFFh {last word all 1's] 
if ^ifio A -^ifii then {compress only if there are all O's or all 1 's] 

return words[ ],top 
end if 



(f'g «- words [top - 1] & COOOOOOOti : 
ip' <- words [top - 1] & COOOOOOOti : 



OOOOOOOOh {O's fill] 
40000000h {rsfill] 



if(</3o A tp'g) V (pi A ifi'^ ) then {previous word is the same filf] 
top <- top - 1 

words [top] <- words [top] + 1 
return words[ ],top 
end if 

'ri{(fQ A i^'j) V {(fi A ifi'g) then {previous word is a different fill] 

return words[-],top 
end if 

w <~ words[top - 1] 
if (fii then 



else 

w <-w& 7FFFFFFFh 
end if 

if w = A BitCount(w) = 1 then 
top <— top - 1 
if If I then 

words [top] ^ 40000001h {r's fill] 
else 

words[top] <- OOOOOOOlh {O's fill] 
end if 

if BitCount(w) = 1 then {check dirty bit] 

words[top] <- words[top] | (I + TrailingZerosCw)) « 25 
end if 
end if 

return words[ ],top 
end procedure 

(b) Compression algorithm. It tries to "merge" the literal that is in the last word 
of the compressed bitmap with the previous word. 



procedure PerformOperation(5i ,52, *) 
S.top ^ 

while HAsMoRE(5i.words[ ]) a HAsMoRE(5'2.words[-]) do 
S.top <- R.top + 1 
i?.words[R.top] <- 

NEXTLlTERAL(Sl.WOrds[]) * NEXTLlTERAL(S2.W0rds[-]) 

CoMPRESs(S.words [■], R.top) 

{check if we just created a fill] 

if R.words[R.top] & SOOOOOOOh = OOOOOOOOh then 

s <- minjLENGTH(5i.words[-]), LENGTH(S2.words[ ])) 
if i > then 

Skip(5 1 .words[-], s), SKip(5'2.words[-], s) 
i?.words[S.top] <- R.words[S.top] + .v 
end if 
end if 
end while 



17: {copy remaining words] 

18: if * is bitwise OR or XOR then 

19: if HasMore(5 i.words[ ]) then 

20: append to R all the remaining words of S 1 

21: else 

22: append to R all the remaining words of S2 

23: end if 

24: else if ★ is bitwise AND-NOT then 

25: append to r all the remaining words of 52 

26: end if 

27: return R 

28: end procedure 

(c) Bitwise operations between two compressed bitmaps. "*" indicates the 
desired binary operation. The algorithm iterates over both word arrays and 
performs the operation represented by ★. HasMore() indicates if we reached 
the last word, while NextLiteral() extracts the next literal. When applied to 
a fill, NextLiteralO returns all the literals represented by the fill: it stores 
the position of the current literal within a fill, making it possible to "skip" a 
specified number of blocks by calling Skip(). Length() gives the length of a 
fill represented by the word w (i.e., w & OlFFFFFFh), or for literals. 



Figure 3: The Concise algorithm. Figure [(a)] describes how to create new compressed bitmaps (indicated with the array words[-]) by "appending" integral numbers 
in ascending order — namely we can only add integrals that are greater than the last appended one. Figure [(c)] describes how to apply AND/OR/XOR/AND-NOT 
operations over compressed bitmaps. Finally, Figure [(b)]is used by both Append and PerformOperation to compress bits. 
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Table 1: Memory footprint analysis among standard Java structures from the package java.util, WAH, and Concise 



Data Structure 


Distinct? 


Sorted? 


Bytes/Item^ 


Explanation 


ArrayList 






4 


Simple array. It is the simplest data structure. It is internally represented by an array of pointers (4 
bytes each) to Integer instances. 


LinkedList 






24 


Linked list. Each element of the list requires 4x3 bytes (4 bytes to point the Integer instance, 4 bytes 
to point the previous element, and 4 bytes to point the next one), plus 8 bytes used by Java for each 
object instance, and 4 padding bytes. 


HashSet 


/ 




>3() 


Hashtable. Each element requires 4x4 bytes (4 bytes to point the key — the Integer instance — , 4 
bytes to point the value — not used in our tests — , 4 bytes to point the next table entry in case of 
collision, and 4 bytes to store the hash value of the key), plus 8 bytes internally used by Java for the 
table entry. Moreover, we require an array of pointers (4 bytes for each element), considering that 
a hashtable must be greater than the maximum number of allowed elements in order to reduce the 
number of collisions. 


TreeSet 


/ 


/ 


32 


Self-balancing, red-black binary search tree. Each node of the tree requires 4x5 + 1 bytes (4 bytes 
to point the key — the Integer instance — , 4 bytes to point the value — not used in our tests — , 4 bytes 
to point the left node, 4 bytes to point the right node, 4 bytes to point the parent node, and 1 byte for 
the node color), plus 8 bytes internally used by Java for the node object, and 3 padding bytes. 


BitSet 


/ 


/ 


1/8 ^ 2^8 


Uncompressed bitmap. Each integral value is stored in a bit. In the worst case, we need a long 
sequence of zeros and then a word to store the integral. If we only consider positive integral numbers 
represented in two's complement over 32 bits, the greatest number is 2^' - 1. In this case, we need a 
bitmap of 2^* bytes. In the best case, all integers represent a sequence of consecutive numbers, thus 
requiring only I bit on average. 


WAH 


/ 


/ 


-0^8 


In the worst case, namely when numbers are very sparse, we need a literal word to store the integer 
(4 bytes) and a fill word to store a zero sequence (4 bytes). In the best case, all integers represent a 
sequence, thus requiring only 1 fill word (4 bytes) to represent all of them. 


Concise 


/ 


/ 


-0-4 


In the worst case, namely when numbers are very sparse, we store each integer in each mixed fill 
word (4 bytes). In the best case, all integers represents a sequence, thus requiring only 1 fill word (4 
bytes) to represent all of them. 



"Please note that each Java object requires at least 8 + 8 bytes of memory: 8 bytes to represent information that are internally managed by the Java Virtual 
Machine (JVM), while user-defined object data should be aligned to a multiple of 8 bytes — in case, memory is padded with O's. Moreover, in standard Java 
collections (namely any class implementing the Collection interface, such as ArrayList, Linl^edList, HashSet, and TreeSet), integral numbers are stored within Integer 
instances. This means that each number to store requires 16 bytes (8 for internal JVM data, 4 for the integer, and 4 for padding) in addition to those reported in 
this table. Instead, BitSet, WAH, and Concise directly stores integers within an array of ints, hence avoiding to "waste" this additional space. 
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Figure 4: Time and memory measurements. "Compression" means the ratio between the number of 32-bit words required to represent the compressed bitmap and 
the cardinality of the integer set. "Density" means the ratio between the cardinality of the set and the number range. "Max/Cardinality" means the ratio between the 
maximal value (i.e., the number range) and the cardinality of the set — that is, the inverse of the density. Time measurement are expressed in nanoseconds. Since the 
experiments were peii'ormed in a multitasking environment, to have a more accurate time measurement, each experiment was performed 100 times, and the average 
reported. 
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law, many types of data studied in the physical and social sci- 
ences can be approximated with a Zipfian distribution |9]. 

Figure |4a] reports on the memory occupation of one ran- 
domly generated set. It demonstrates that, according to 
when density is below 0.05, WAH starts to compress. Since 
Concise is able to compress the same bitmaps that WAH can 
compress, both algorithms start to compress after the same den- 
sity threshold. However, CONCISE always has a better compres- 
sion ratio, which tends to be half of that of WAH when the den- 
sity approaches zero. In Figure |4b] results are very similar, but 
it is more evident the compression ratio of WAH is twice of that 
of Concise as the data becomes more and more sparse. As ex- 
pected, uncompressed bitmaps (BitSet) continue to increase as 
the maximum integer value grows, while other data structures 
are not affected by the data density. 

Figure |4c] reports on intersection time of two sets, namely 
the time required for the identification of shared numbers be- 
tween two sets. We do not show results for union and set dif- 
ference because they have demonstrated a very similar compu- 
tation time. For Java classes, intersecting corresponds to call- 
ing Collection. retainAIIO and BitSet.and() methods. Notice that 
WAH and Concise are always faster than Java structures, apart 
from BitSet that is far much faster when the set is not sparse. 
However, BitSet performance drastically decreases when data 
becomes very sparse. Again, Java data structures are not af- 
fected by the density. In our experiments, we also noted (as 
expected) that the intersection time changes linearly with re- 
spect to the cardinality of the set. Similar considerations can 
be done for Figure|4d] Notice that other curves can be justified 
in the following way: lists (ArrayList and LinkedList) requires a 
full set scan to perform the intersection, binary tree (TreeSet) 
a logarithmic time search and hashtable (HashSet) an almost 
constant time search of shared elements. 

In turn, we analyzed the time to add single numbers to a 
set. In Figure |4e] we report on the append time, namely on the 
addition of a new number that is strictly greater than the ex- 
isting ones. Formally, S U [e] when Ve' e S : e' < e. This 
corresponds to a sequence of calls to Java Collection. add() or 
BitSet. set() where numbers are first sorted. The append oper- 
ation is the fastest way to add numbers to Concise and WAH 
bitmaps. Instead, sorting does not influence the performance 
of other data structures. Notice that the append time is con- 
stant for all structures and, as we observed in our experiments, 
it does not greatly change as cardinality grows. However, for 
very sparse data, the BitSet class spend most of its time in allo- 
cating memory, hence resulting in poor performances. 

Finally, we analyzed the time to remove a single number 
from a set. In Figure|4f|we indicate the corresponding execution 
time. Since both WAH and Concise do not explicitly support 



removal of single elements, we implemented it by performing 
the set diff'erence between the given set and a singleton. Note 
that the same thing can be done for the addition of new integers 
when the append operation is not possible, by just performing a 
union between the set and a singleton. In this case, the reduced 
size of the compressed bitmap causes that Concise is much 
more faster than WAH on sparse datasets. 

4. Conclusions 

Because of their property of leveraging bit-level parallelism, 
computations over bitmaps often outperform computations over 
many other data structures such as self -balancing binary search 
trees, hash tables, or simple arrays. We demonstrated, through 
experiment on synthetic datasets, that bitmaps can be very effi- 
cient when data are dense. 

However, when data become sparse, uncompressed bitmaps 
perform poorly due to the waste of memory. In this paper we in- 
troduced a new compression scheme for bitmaps, referred to as 
Concise, that is a good trade-off' between the speed of uncom- 
pressed bitmaps and the required memory. Indeed, Concise 
outperformed all analyzed data structures in terms of memory 
occupation, as well as WAH, the best known compression algo- 
rithm that allows for set operations directly on the compressed 
form. As for computation time. Concise also outperformed 
classical data structures for set operations. 

However, accessing individual elements can be expensive 
for both Concise and WAH. If random access is more common 
than sequential access, and the integer set is relatively small, 
classical data structures may be preferable. 
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